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SYSTEM FOR AUTOMATED GENERATION method provides for selecting some of those media elements 

OF MEDIA in response lo a request for media programming, and select- 
ing a temporal organization for the media elements. 

RELATED APPLICATIONS However, the temporal organization is not dictated by the 

Tins application claims priority from U.S. Provisional 5 selected information that regarding each i of the media clc- 

Patent Application No. 60/042,564, filed Apr. 1, 1997, which ments ; The system selects and orders the media elements 

is hereby incorporated by reference in' its entirety. according to the data m the request, and according to 

information, such as permitted transitions, regarding the 

HEKD OF THE INVENTION media elements. The system prevents a user from selecting 

10 individual media elements. The media elements are then 

This invention relates to a method and computer- assembled into media programming, 

implemented system for creation of audiovisual program- In aQOthcr aspect of lhe irjvcnlioQ , a meLhod ^ providcd 

mm S' for verifying viewing and comprehension of a unique media 

BACKGROUND OF THE INVENTION 1 program. The method includes providing, in the unique 

15 media program, a unique sequence of cues. The method 

There have been recent substantial advances in the capac- includes receiving from a viewer of the unique media 

ity to design customized audio visual programs for specific program information relative to said cues, such as responses 

purposes from a library of existing video clips and audio to questions included on a questionnaire, or in response to a 

elements. Customization of audiovisual programing is use- telephone call made by the viewer. The received information 

ful in many applications. For example, in advertising certain 20 is then compared to the sequence of cues to determine 

products, and in particular automobiles, one promotional whether or not the program was viewed, and the level of 

technique is to prepare promotional videotapes which are comprehension by the viewer. 

sent to potential customers on their request. The desirability i n another aspect of the invention, a method of creating 
of customizing such videotapes to demographic or other audiovisual programming from stored audiovisual media 
characteristics of individual consumers arc of course sub- 25 elements is provided. In a first step, from a database con- 
stantial. Health care practitiooers and managed care entities taiaiQg information concerning the audiovisual media 
have begun to provide instructional videotapes to patients elements, certain audiovisual media elements are selected. A 
with information regarding managing of various diseases temporal sequence for the selected elements is designated, 
and conditions- Customizing of such information to the Transitions between the media elements are automatically 
disease and condition of the individual, and demographic 30 selected, 
characteristics of the individual, such as age, income, edu- 
cational level, psychograpfuc characteristics such as per- BRIEF DESCRIPTION OF THE FIGURES 
ceived wellness and willingness to change behaviors, and FIG. 1 is a schematic diagram of a system according to the 
other factors, would be valuable for increasing the effec- invention 

tiveness of such video tapes in communicating the informa- 35 mQ 2 {& a schcmalic diagram Seating iu f ormatiou 

tion to the recipient. contained in a database according to the invention. 

In accordance with present technology, it is possible to FIG. 3 is a schematic diagram indicating the organization 

create and store a library of brief video clips, and provide a of a d J tabase accordiug t0 tbe mV ention. 

database of information regarding these clips. However, in , . . • t - c . i 

, . . > . , , , • ■ an FIG. 4 is a schematic diagram mdicating software ele- 

accordance with the present technology, a human editor 40 , - 1L * ■ - . . • ■ 

, . , . \ . . c P y ' . , . ,. - ments in the system according to the invention, 

must make the ultimate selection of individual clips, and 3 , 

provide the final editing decisions, creation and selection of FIG. 5 is a flow chart showing steps in the assembly ot an 

transitions so that there is a smooth visual and audio edlt decisio » lisl ™ the method according to the invention, 

transition between adjoining clips in the program, and HG. 6 is a flow chart showing steps in the assembly of 

checking of the content of the clips to determine that there 45 programming in a method according to the invention, 

is proper coverage of the appropriate subject matter in an FIG. 7 is a flow chart showing steps in a method according 

appropriate sequence. Automating of this editing process to the invention of verifying viewing of a program, 

would make possible substantial flexibility and new possi- DETAILED DESCRIPTION OF A PREFERRED 

bihties tor creation or audiovisual programming. EMBODIMENT 

Once videotapes have been provided to the user, it is 50 n c * iL • _. ■ * , . -m a 

difficult lo verify whether or not (he user has viewed the . R ° l '™& to HO. l » ' here ls ^P?*? a ^ {em 10 ««>«»- 

program. Even if the program has been viewed, the level of !°* l ° Uk System 10 mcludes a computer 20 

comprehension is difficult to assess. including processor 30, input devices 40, output and display 

devices 50 and memory devices 60. Computer 20 may be a 

It is accordingly an advantage of this invention that the 55 conveotional petS0Qal compute r with suitable processor 

disadvantages of the prior art may be overcome. spee( , an( , memory capacity aQd spee d. Computer 20 com- 

Additional advantages of the invention, and objects of the municates in a suitable manner with library 70 of audiovi- 

invention, will become apparent from the detailed descrip- suai clips 80. Library 70 may be stored on any suitable 

tion of a preferred embodiment which follows. storage medium of adequate bandwidth and retrieval time 

qtix/uadv actuc TMUCvmAVI 60 characteristics. At p resent, certain fixed or hard disk systems 

^ SUMMARY OF THE INVENTION proWdc superiof performancc to other optioDS> Digitizcd 

According to a first aspect of the invention, a system and audiovisual clips 80 are stored in library 70. Each audiovi- 

mcthod of creating media programming are provided. A sual clip is suitably identified, such as by the name of one or 

database is provided which contains selected information more files which make up the clip, 

about each of a large number of media elements. The media 65 Database. 

elements may be, for example, audiovisual clips, 'lhe eie- Computer 20 is also suitably associated with a database 

ments themselves are maintained in a suitable library. The 100. Database 100 contains unique identifying information 
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for each clip and has associated therewith additional infor- family of defining values which describe the range of the 

mation often arranged in a hierarchical manner. Referring to target audience in specific demographic and psychographic 

FIG. 2, this information will be referred to herein as a header terms. Additionally, the overall outline of the subject matter 

110. Individual items of information within the header will is contained in this layer and is associated with demography 

be referred to herein as tags. The tags arc divided into two 5 where appropriate. These outlining functions arc secondary, 

general categories, namely control tags 115 and content tags however, to the temporal organizational templates embodied 

120. Content tags 120 contain information identifying the in this layer. Here the instructional designer, or interactive 

content of the clip. The content is often embodied in three author, defines the preferred temporal modes of presentation 

general categories, namely content, i.e., information com- of the universe of assets. For example, the instructional 

municated by the clip, as indicated by block 125, viewer 10 designer might define that the block of programming content 

profiles for viewers most likely to be receptive to the clip, called EFFECTS ON THE HEART is presented across three 

indicated by block 130, and intended use of the clip, fundamental age groups, two levels of detail (summary/ 

indicated by block 135. The content tags contain information cursory and in-depth), both gender specific groups and four 

defined by a suitable system of classification of information. distinct ethnicity components. Within this multi- 

For example, in a database of clips for the assembly of video is dimensional array of program assets, the instructional 

programs concerning medical information, the content infor- designer might also define that the material be presented in 

mation may, in addition to other systems, such as for the preferred sequence of — INTRODUCTION TO THE 

categories of content mentioned above, make use of Inter- HEART, IMPACT OF DIABETES ON THE 

national Classification of Disease codes. In databases CARDIOVASCULAR- VASCULAR STRUCTURES, 

directed to assembly of programs for other topics, other 20 EFFECTS OF DIET, EFFECTS OF EXERCISE, Q&A 

coding systems are used for the content information. Fields SESSION, SUMMARY. 

may be provided to indicate technical complexity, specific Below the instructional design layer are the smaller 
items of information conveyed, demographic characteristics organizational elements which allow for elasticity in the 
of actors in clips, style of script, and other suitable infor- specifics of the implementation of the temporal design. This 
mation. One or more fields may be provided within the 25 is called the MODULE LAYER 320 and in special instances 
content lag to deDnc conLcnt. The system by which the the SEQUENCE LAYER. Fundamental to this layer are 
content information is defined is preferably organized to weighting factors which control likelihood of asset use, and 
provide for relationships among types of information. Such allow for the deployment of elements which are free to float 
relationships may be levels of generality, and other subject temporally in order to accomplish certain transitioos and 
matter relationships. 3u effective deployment of those elements which are slave to 
The organizational structure of the database may be . the temporality functions. These elements as a group arc 
hierarchical, with each layer of hierarchy defining a specific shorter sequentially-patterned program elements of content 
set of organizational principles. Referring to FIG. 3, at the which organize under the temporality principles of the 
highest level the global behavior of the universe of elements higher layer The free floating elements may have various 
is defined. This is equivalent to automating a * loo k-and -feel' 35 linking geometries or parameters at the opening and closing 
function for the entire unit of programming. A given stylistic thereof. Such elements can be used to bridge elements that 
approach 300 can be defined through the use of default cannot themselves be linked because a direct link is either 
values which will be employed, for example, in the treat- disallowed or would involve use of disfavored transitions, 
menl of transitions between adjacent clips. There may be The lowest level of organization is that of the individual 
defined a set or range of transition values, such as a range 40 media elements or assets themselves, 'ihis is called the CLIP 
from hard-cut to 24-frame dissolve. In this example, the LAYER 325. These elemenrs carry tags which define their 
master look-and-feel level of organization might define specific content, such as: DIABETIC HEART, LEFT 
hard-cur as the aesthetically optimal, and therefore the VENTRICLE, DAMAGE TO, HYPERGLYCEMIA, 
default, choice. Numerous other aesthetic or programmatic MALE, AGE 50, TALKING HEAD. The first three content 
choices that affect the delivery and placement of media 45 tags will be noted as being in hierarchical order from most 
elements in the concatenated stream can be defined. . general to most specific. The next two are examples of 
Also at the highest level of organization, typically used in demographic tags, and the final tag is a simple example of 
on-line applications only, there may be provided the viewer/ a tag denoting style. These elements also carry production- 
user interface options which define the ways in which any specific control tags, which, as discussed in more detail 
given class and security level of user will be allowed to 50 below, define such characteristics as allowable exit/entrance 
actively as well as passively interact with media assets. We transitions for both audio and video, 
will call this the INTERFACE LAYER 305. At this level of It is importaut to uote that, in the operation of the system, 
organization, the behaviors of ancillary assets such as pro- there is an inhibition layer between the clip and the search- 
motional segments, information identifying the system, ing mechanism. The inhibition layer assures that the system 
advertisements and news-flashes are defined. These assets 55 does not include in the programming every clip that is 
embody aesthetic, program or instructional design, as well responsive to a query provided by a user. The inhibition 
as market-driven, or viewer defined behaviors. mechanism may be responsive to psychographic character- 
Immediately below this layer is preferably the meta- istics of the user, such as age, level of education, or even 
content layer. This is called the PROGRAM LAYER 310. reason for the query. The tags arc responsive to this type of 
Here are defined the type of assets and the core content 60 information. The inhibition mechanism may be modified 
descriptions of those assets. By way of example, the types of dynamically as the database is mined for suitable assets. The 
assets may be defined as training, informational, and enter- inhibition mechanism may be viewed as in a multi- 
tainment assets. Examples of core subject matter would be dimensional set of psychographic requirements. Clips can be 
"medical", at the highest level, "health management", at a evaluated for their responsiveness in the various dimensions, 
lower level, and "diabetes mellitus", at a still lower level. 65 The system may set, or the user may select, a length of time 
Next in the hierarchy is the instructional design layer, or for the program, and the inhibition mechanism will operate 
TEMPLATE LAYER 315. This layer is characterized by a to limit the total number of clips selected to the selected 
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time, as well as choosing clips according to suitability for 
the viewer and the viewer's purpose. 

Referring again to FIG. 2, control tags define audio and 
video components of clips. The audio and video information 
mav be entirely separate as indicated by block 140, VIDEO 
CONTROL TAGS, and block 150, AUDIO CONTROL 
TAGS. For example, a minimum number of control tags 
would be, for each of audio and video, transition points 145, 
155 an optimum transition point from the beginning of the 
clip, a first possible transition point from the beginning of 
the clip, a last possible transition point from the beginning 
of the clip, an optimum transition point from the end of the 
clip, a last possible transition point from the end of the clip, 
and a first possible transition point from the end of the clip. 
Each of these points may be defined in time, by video 
frames, by tags of clips that may be the subject of the 
transition, or by content parameters. These points may be 
defined by human post -production editors entering such 
information into the database in response to suitable prompts 
contained in the software. These points may also be defined 
by truth-table or expert-system defaults, which may or may 
not reference either video or audio content. The variable 
beginning and exit points may be referred to. as creating 
fuzzy extremities on the clips. This is believed to be a novel 
feature as compared to existing databases of audiovisual 
assets, and lends significant flexibility to the process of 
creation of transitions. Numerous optional control tags can 
be used, such as, with reference to video information, a 
luminance range 160 for the opening and closing of the clip, 
preferred or required transitions, including length and type 
of transition, a dominant chroma value for each of the 
opening and closing of the clip, start and end points for 
dialog and action. Wiih reference to audio information, 
control tags may include key of music 170, tempo of music 
175, preferred transitions and other information. This type of 
information can be derived by suitable software reviewing 
the data stream before or after the clips are concatenated, or 
in advance of the assembly process using clip analogs or 
proxies containing suitable information from the tags. 
Creation of DaLahase. 

Ihe database is created by identifying each clip or other 
asset and defining values of the control tags and content tags 
for each. Values of the various control tags and content tags 
may be defined by a user, either during development of the 
script for the clip or upon completion of the clip. For 
example, the program may include screens prompting a user 
to select a value for each control tag from a menu of options. 
Different screens may be provided for different users, such 
as dialog editors, audio editors, and video editors, to enter 
values for appropriate control and content tags. 
Alternatively, values of various tags may be created auto- 
matically by truth tables or decision-capture systems, or 
other automated techniques, either with or without huuiau 
assistance. Such systems may act from information derived 
from analysis of existing clips using image-recognition 
software, from analysis of scripts entered into text, or from 
other information provided by human editors. 

By way of example only, a system for creation of a 
database for use in connection with creation of medical 
programming will now be described with reference to FIG. 
4. The system may include five primary components, each of 
which will be provided with its own user interface: 

1. A production interface 405 which provides an interac- 
tive means of importing a video clip into the system and 
recording the transition production information. 

2. A coding interface 410 to allow the entry of the medical 
and patient selection codes which apply to the clip. 
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3. A patient profile entry interface 415 and decision list 
processor. 

4. A decision/transition list editor 430 with built-in tran- 
sition previewer for both audio and video components 

5 of clips. 

5. A production player/recorder which embodies an 
engine for both audio and video transitions. 

Each of the foregoing components operates as an inde- 
pendent entity in the exemplary system. Each component is 

10 started by the user selecting an icon from an appropriate 
folder. The user interface is preferably be graphical in 
nature. Actual keyed data entry is preferably kept to a. 
minimum where most entries are made by selecting an 
option from a set or an entry from a list. 

15 The Production Interface 

The production interface 405 is the interface designed to 
provide a structured, interactive means of importing a video 
clip into the system. A standard container, such as OMF, may 
be utilized as a standardized vehicle for the transmission of 

20 the proprietary tags. When selected, the user will be pre- 
sented with a screen containing fields for the entry of the 
following: 

Clip ID — This will become the primary key or file name. 

Source — Where the clip will be read from. Options may 
25 include DAI" tape or network. 

The user will also be presented with one or more inter- 
faces to enter the audio/video transition fields and the coding 
(medical/patient demographic) information. Before being 
allowed to exit the process, defaults will be applied to the 
30 transition fields not specified by the user. The user may be 
required to preview the clip just imported. 

The Coding Interface 

The coding interface 440 is the GUI designed to perform 
the entry of the medical and socio -demographic selection 

35 codes which apply to the clip. These include the I CDs for 
which this clip is applicable, the socio-demographic and 
medical attributes of those patients assumed to be the most 
likely potential viewers of the clip, and any special fields 
which when present in the client/patient profile will cause 

40 this clip to be selected. All fields may be selected from 
preestablished lists or groups displayed as "radio-buttons". 
As the ICD set may require the designation of hundreds or 
thousands of codes, a suitable method of use of the hierar- 
chical nature of the ICD structure may be utilized to simplify. 

45 the selection process. These may be selected during the 
script outlining or scripting process by the use of menus or 
existing fields incorporated in the scripting software. 
The Patient Profile Entry Interface 
The Patient profile entry interface 415 is the GUI 

50 designed to perform the entry of the general, medical and 
socio-demographic codes for the specific patient for which 
the product is to be produced. The general section may 
require the most data entry including patient name, address, 
social security number, insurance billing numbers, referring 

55 entity identification and any text which is to be added to the 
final video. Systems, such as those used in the health care 
industry, may be employed to extract relevant information 
from existing patient records. In the case of on-line use, the 
medical and socio-demographic sections will be "click to 

60 select" entry similar in nature to the entry process performed 
by the coder when the clip is imported. The result of this 
process will be to create an entry in the patient database 420. 
The information is then forwarded to the editor program 
425. 

65 Based on the topic requested, such as a condition which 
the patient has been diagnosed with, and the demographic 
information, editor program 425 will, as discussed below, 
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produce a recommended decision list, or preliminary edit Editor Program. 

decision list(EDL) file for further processing. Subsequently, Programs are created from clips by an editor program, 

the final EDL is created in an evolutionary manner. At first, The identify and order of the clips that comprise a 

a sample order of clips, without transitions, is analyzed. The program may be defined by a variety of methods varying in 

order and identities of the clips is then revised. When the 5 the degree of control exercised by the individual user. At the 

order and identities of the clips has been finalized, the level of least control by the user, the user defines only the 

transitions are computed. The transitions are then executed overall functional requirements for the program. For 

and inserted into the stream by use of a system of removal example, in the assembly of programs for education of 

of portions of the tag extremities used in temporal transi- patients about health issues, the user may define only the 

lions. demographic characteristics of the intended recipient of the 

The List Editor/Prcvicwcr program and the information to be conveyed, e.g., managc- 

The list Editor/Previewer 430 is the GUI designed to ment of a particular condition. In one embodiment, the 

provide the production editor with the ability to change, add system may simply select appropriate clips from the data- 

to and preview the EDL that has been produced by the editor base and apply suitable transitions. Alternatively, an expert 

program 425 in response to information entered in the system included within the editor program selects and orders 

Patient Profile Entry Interface 415. 15 one or more suitable templates as shown in block 505 of 

The Production Player FIG. 5. The template may incorporate the demographic 

Each clip, clip component, or other media asset, is stored characteristics and information to be conveyed, or may 

in three distinct segments: contain only sequential requirements for the underlying 

1. lead-in segment structures used in the program. Using the requirements in the 

2. main body 20 selected templates and the demographic characteristics and 
3 ex j t seg llieu t other requirements specific to the program, as well as the 
Transitions can only be performed on the lead-in and out inherent structural characteristics of the modules and 

segments. The production player 435 is launched with the sequences, the expert system or truth table then selects and 

name of the patient EDL to be. played. The EDL will be orders oue or more suitable modules aud floater sequences, 

analyzed and each transition rendition time estimated. The 25 as shown by block 510 of FIG. 5. Using the requirements in 

production player will then process each command in the the selected modules and the inherent structural character- 

EDL. Most commands, which require very little processing, istics of sequences, the expert system or truth table selects 

cause a more detailed command to be written Lo the player and orders suitable sequences, as shown by block 515 of 

FIFO which controls the player output thread. Other process FIG. 5. Finally, using the requirements in . the selected 

intensive commands, such as a transition, will be rendered, 3u sequences and the inherent structural characteristics of the 

stored lo disc and the detailed command with its disc clips, the expert system or truth table selects and orders 

location is added to the player FIFO. As each EDL command suitable clips, as shown by block 520 of FIG. 5. The expert 

is processed, the remaining estimated time flow is modified system or truth table also maizes transition selections at the 

until the program determines that future transition rendering beginning and end of the audio and video portions of each 

will be able to be completed before the player output thread 35 clip, as shown by block 525 of FIG. 5. At any level in this 

requires it. At this point, the player output thread will be process, if there is an incompatibility, the expert system or 

started taking its input instructions from the player FIFO. truth table may discard an element aud replace il with a 

The player output thread will operate asynchronously, trans- substitute element, or employ suitable transition methods, 

ferring each data component specified in the player FIFO to For example, if it is not possible to assemble acceptable 

the appropriate hardware component, such as video and 40 sequences in the manner dictated by the modules under the 

audio decoder modules, text merge processor, etc., whose template, the expert system or truth table may discard one or 

output drives the device for recording the program, such as more modules and select alternatives that meet the require- 

a videotape deck. ments of the template. 

Upon completion, the required managerial and billing The tags may include information regarding such 

information will be generated and stored in an appropriate 45 attributes as luminance, chrominance, music tempo and key, 

database. and colors, at the beginning and end of each clip. 

The transition process will require the identification of Alternatively, a MIDI record of the audio portion of the clip 

both an exit and lead-in point, and a transition function may be analyzed. The editor program may apply expert 

specification. This pair of fields are refined into a single system software or truth tables to determine whether a direct 

recommended field as defined above, and then the fiual 50 transition between any two clips meets aesthetic require- 

recom mended EDL is obtained. These fields are contained in ments. If not, the system may identify a suitable type of 

the preliminary EDL which is being played. The appropriate transition for both cups, add a bridge between the clips, or 

time positions are located in both the exit and lead-in determine that one of the clips must be discarded. For 

segments. The first frame of each segmenL is decoded into example, if the keys of the music in two adjacent clips are 

separate buffers and then the transition function is applied to 55 incompatible, the system may add a bridge consisting of a 

each matching pixel of both images to produce the third burst of percussion in the audio portion of the program. . 

rendered frame buffer. A similar process is also applied to The viewer or user may directly select templates, 

the matching audio sample points associated with the cur- modules, or sequences, depending on the degree of control 

rcntly rendered frame. The video and audio is then desired. Decisions regarding cither specific production deci- 

re -compressed and written to the output transition file. 60 sions or global stylistic choices or content or order can be 

Values of certain control and content tags, such as key or captured by the system by, in addition to expert systems, 

tempo of music, may be determined by suitable software truth-table or other decision tree prog ramming specific deci- 

from the clip. sion collection fields provided to content developers. Expert 

Il is to be emphasized that the foregoing system is merely systems, truth tables and other systems may be used to create 

exemplary, and does not limit the scope of systems that may 65 the tags associated with the clips. 

be implemented within the scope of the inventions described Use of expert systems, or decision -capture systems is of 

in this application. interest because the system might organically evolve siylis- 
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tic tendencies which might mimic or mirror those of creative can still be employed to generate useable dissolves. Using 

professionals. Templates might be provided to content pro- this technique, the full decode process caa be further 

viders or even to end-users, which would allow a specific reduced, and thus accelerated, by processing this luminance 

style of editing, audio cutting or mixing, or program forma- in averaged blocks of pixels. These pixel ' blocks are 

lion; perhaps a template maybe provided associated with an 5 employed in the encode process. Even without the use of 

individual editor. Such decision -capture systems already MPEG, it is possible that some or all of these shortcuts might 

exist for other uses which could be adapted to the assembly be effectively employed to accelerate the creation of dis- 

of audio and video. solves without use of a full decode/recode cycle. 

In one embodiment, the user creates queries depending on Viewer Database, 

the qualities desired for the organization levels to be iden- 10 Separately from the database of media assets described 

tified. The editor program then identifies suitable templates, above, there may be provided a database of viewer infor- 

modules or sequences based on the queries. The relative mation. This database defines the identity of each viewer, 

importance of different items in the queries may be including name and address information, and may also 

weighted. The weighting may be supplied by the user in include social, economic and medical data regarding the 

response to suitable prompts, or may be generated automali- is viewer, the history of items viewed by the viewer, and a file 

cally by the system based on existing programming. Using of preference information for the viewer. In an on-line 

suitable relationships among data items, the user may be environment, the viewer may be prompted to provide viewer 

presented by the editor program with one or more templates, identification information which may be entered into a 

modules or sequences that represent a fundamental database or may be retained temporarily as a viewer data 

response, another set that represent a secondary response, a 20 record for use in creation of customized programming, 

third set that represent a tertiary response, and so forth. Verification of Viewing. 

Assembly Program. The verification of viewing of the delivered video pro- 

Once the set of clips has been defined, the assembly of the gram is difficult to accomplish. It is important for the 
clips from the library takes place. This is accomplished by well-being of the viewer when the program contains tbera- 
an assembly program. Referring to HG. 6, the assembly 25 peutic or training video programming, and useful for all 
program reads a file of clip identifiers and transition infor- content to assess the effectiveness of delivering information 
mation created by the editor program as shown by block via customized video programming. An effective system 
605, and calls to the lihrary for each identified clip as shown would: (a) allow the distributor of the programming to know 
by block 610. The identified clips are then transmitted from if no effort had been made on the part of recipient to watch 
the library to a suitable destination. The assembly program 30 the delivered material; (b) provide disincentives for fast- 
uses the transition information in assembling the clips into forwarding and skipping through the program; (c) allow for 
the program. If the clips are transmitted to a suitable digital the confidential verification of the particular program 
to analog converter, the clips may be assembled into a watched, thus allow for the confidential generation of 
program on videotape without compression. However, in viewer-comprehension testing; (d) provide incentives, per- 
order to assemble the program as a digital audiovisual 35 haps monetary or material, for the successful viewing of the 
program in memory, or to transmit the program over tele- material; (e) maintain ; confidentiality of the viewer and/or of 
phone Hues or the Internet, data compression is required. the precise content of the delivered program. The following 
Suitable data compression techniques and standards may, be viewing verification system performs all of the objectives 
used. outlined above. 

The assembly program may also dictate particular tran- 40 Referring to E1G. 7, the program software creates an 

sitions. For example, asymmetrical transitions in audio are identifier sequence of cues, as indicated by block 705. The 

advantageous. A leading edge audio transition which is cues may be numbers, letters or other indicia, such as colors, 

exponential and short, and a trailing edge transition in audio shapes, sounds or combinations thereof, unique to each 

which is linear and long is preferred. The video transitions viewer and each program. The sequence may be derived 

need not match the audio transitions. 45 from such information as the viewer's ID number, such as 

The following is a technique which is believed by the the Social Security number or other unique record locator 

inventor to be suitable for improving performance of com- contained in the patient/viewer record database, the unique 

pression of the programming, for use in compression segment and/or program ID numbers or the content of clips 

formats, of which MPEG is an example, which achieve or other portions of programming delivered to the viewer, 

performance using predictive compression which includes 50 Regardless of the method employed, a unique key sequence, 

both forward and backward looking frames. The technique or unique sequence of cues, which may be alphanumeric or 

is the elimiuatiou of the complex frames from the MPEG comprise a series of images, icons, colors or souic cues, is 

stream using the last-in and first-out video marker tags to created. This sequence is unique to the customized video 

determine the amount of leading and trailing video which program. The elements of the sequence are recorded within 

might, in a worst-case scenario, participate in a visual effect. 55 the program, as indicated by block 710. The cue are pref- 

By eliminating these (P and B) frames, it is possible to erably spread throughout the temporal sequence of the 

employ a partial decoding of the (Huffman) algorithm, rather viewing experience one or more cues at a time. By way of 

than a full decode/encode cycle. Additionally, the elimina- example, numbers in the sequence may be superimposed 

lion of these PB frames allows impunity in the concatenation over picture elements using character generation and keying, 

points employed. This freedom is bought at the price of 60 The picture sequences which underlie these characters may, 

increased bandwidth requirements, but only for small seg- or may not address the superimposed characters/numerals, 

ments of video rather than entire clips. In the case of audio cues, it is of course more likely that a 

Another technique is applicable to compression formats, specific sequence, such as a talking head, would be 

such as MPEG, where the chrominance data is subsidiary to employed to communicate the cue. A special case of the cue 

the luminance data. This technique involves emulating the 65 is the use of actual questions, usually relating to viewer 

MPEG transitions by extracting luminance -only data. comprehension of the presented materia] and having mul- 

Because the chrominance data 'rides on' the luminance, this tiple choice responses. 
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As indicated by block 715, the viewer is provided with a is, at minimum, the veritication of active viewing of the 

method for response. The response method includes suitable temporal stream of information. The responses are com- 

method to record these numbers, characters, colors, shapes pared to cues and expected responses, as indicated by block 

or other cues contained within the sequence. For viewers 725. At maximum, the information is also capable of veri- 

who arc receiving the programming online through a modem 5 fyiog time of viewing, comprehension of content, attitudes 

or Internet link, this might be a window which remains open regarding content (by asking for reactions in some or all of 

and active during the playing of the video sequence, or the fields), as well as motivating viewers with the possibility 

which becomes open and active upon the presentation of the of prizes, secrets, entry into further lotteries or drawings, or 

appropriate superimposed strings, which allows the viewer other perceived rewards. The information can be used to 

Ihe opportunity to record that sequence. For videotapes io instruct viewers to view the material again, make calls or 

delivered to the viewer, a card may be delivered together other responses to the system, view or read additional, 

with the videotape. The card may contain boxes to be supporting or further material. All of this, significantly, can 

checked, blanks to be filled in, or choices of opaque cover- be done while maintaining complete viewer privacy. It may 

ings to be removed. Suitable instructions will be provided in also be important that some of the feedback can be local to 

the video for the viewer to scratch off or mark the card in an 15 the viewer, while other portions of the response -strings may 

appropriate place. involve system-collection of responses. 

In the case of scratch-off cards, or other pre-prepared Techniques may be used for distinguishing between incor- 

cards, it is significant that colors and/or shapes might be rcct responses resulting from a failure to view or comprc- 

employed rather than known characters, or as elements in hend the programming and an error in completing a response 

known characters, like the commonly employed segments of 20 card. Techniques may also be used for identifying the most 

characters used in LED displays. Some action is required of likely error in optical character recognition review of 

the viewer to cause these characters/signs to be recorded/ . responses. The sequence of numbers, letters, or other visual 

transferred to the recording area. These cards might bear or audio cues which is generated for and inserted in the 

sponsorship or advertising data and this data might also programming is also used to generate computer-readable 

generate' or contain further fields. These cards, or on-line 25 printed information, such as bar code, corresponding 

components might come from a third-party source. For uniquely to that sequence. Customized codes may be devei- 

instance the on-line window might be generated for, or by, oped to correspond to cues other than alphanumeric char- 

an advertiser or vendor. A paper-based recording space acters. The printed information is applied to a return card 

which might be contained in a booklet, magazine, newspa- supplied with a videocassette on which the programming has 

per or as part of the packaging itself, may also be generated 30 been recorded. The printed information may be encoded, by 

by or for an advertiser or vendor. Such a recording space altering the order of codes or including additional codes or 

may be a peel-off videocassette label or a peel-off, tear-off, other known techniques, for security purposes. Correct 

scratch-off, or stick-on component of the videotape packag- answers to question fields may be embedded in the bar coded 

ing or its electronic analog. This component might also string. 

contain one or more other characters, colors, icons or strings 35 Acceptable tolerances for error within which the respon- 

which might be included in the final string, or used sepa- dent will be deemed to have viewed and/or adequately 

rately. This component might also be custom-generated as comprehended the programming may be determined for 

part of the custom packaging. For instance, the sequence each position of the character string. Numerous consider- 

ABC-might already occupy a place in the sequence- aiions may be used in determining these tolerances. Exem- 

recording fields such as AKC-xxx-xxx-xxx. The ABC-might 40 plary considerations are the following: (1) the first position 

be visible or might require manual intervention or problem of the string is prone to a heightened incidence of general 

solving to reveal. This problem solving might be related to user error; (2) fields which call for the answering of ques- 

the material presented on the video. For example, a scratch- lions or scratching off of a layer of obscuring material are 

off grid of multiple choice questions, or an on-line emulation subject to a heightened incidence of general user error; (3) 

of such a grid, presented by the programming or packaging 45 for positions known to include only a limited number of 

itself might yield a unique sequence of characters or signs. possible responses, statistical weighting can be employed to 

The final uses of this character string are manifold, but the assign unclear responses into one of the known responses, 

overarching intention is to provide a motivational arena for e.g., if the possible responses are to write out one of the 

the recording of this verification data. The verification data, capital letters A, B, C and D, and an OCR program reads a 

unless used in an on-line environment where the respondent 50 response as the numeral 0, the response will be assigned as 

is known to the system, usually contains fixed identifying the letter D; (4) ia a system employing scratching off of 

data used to coufirm the identity, of either the viewer/ selected areas on the face of a response card, columns which 

respondent or the specific program viewed. It may also contain more than one removed field where one field is 

include, or consist solely of, data sLrings generated in correct and not more than two fields are removed. A 

response to queries. It may further include gaming fields and 55 weighted tolerance system may also employ other strategies, 

or third-party sponsored information. For example, a parity field may he derived from a modulo- 

In the case of traditional videotaped or other passively cycling of the total character string, where the base number 
consumed media, the response data can be collected swiftly' of the modulus might be the number of total possible 

by 800-numbcr, or standard toll-line, call-in to an automated characters potentially employed in a single field of the 

collection system, which has been programmed in a suitable 60 string. By way of further example, a system may be used 

manner to collect response data. The response data can also which causes an known and repeatable limitation to be 

be collected by mail-in and even correlation with sealed- placed on the possible use of a given character derived from 

envelope or other hidden message systems such as peel-open the earlier or other character in the string. The source of such 

and scratch off answer fields which provide interim inslruc- a check character is from within the printed information, 

Lions. For viewers with suitable computer equipment, the 65 whereas me checked characters are from the handwritten or 

response data can be returned by electronic mail. The receipt scratch-off fields. For example, the presence of an even 

of responses is indicated by box 720. The end result though number in the first position of a string might dictate the use 
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of an odd number in the second field. Alternatively, a contrasts to traditional databases in that it mines non-linear 

template could be created which would dynamically or in a assets for presentation in a linear manner, the linear manner 

predetermined manner limit the possible characters or other not being contained within the assets. The use of a modera- 

cues which could be employed in each position of the tion layer that limits the. assets responsive to the query that 

generated string. This data can be used to optimize the OCR 5 are actually presented to the user contrasts with traditional 

function of each position of the response string, as well as database organization, in which all items responsive to a 

to dynamically shift the error tolerance on a position-by- 4 uer y arc presented to the user. 

position basis ^he system also provides for the automatic creation of 

Bv creating a master parity character and a known pattern new programming, uniquely configured to the requirements 

in the string of characters, it is many times more probable io ot ! he viewer but which may be viewed passively. In 

,, . ■ | « t , , , , ■ • 1 j - , contrast, in such fields of video games, while the creation ot 

that a smgle mcorrectly read character can be isolated by the ■ .u • * ■ * > ■ u 

, 6 c Cil _ . . lC 7 unique programming occurs, the viewer must interact with 

logical cross-reference of the two strategies set forth above. ^ \ ^\ order * 0 create (he programming . 

For example, if the parity check « incorrect by one value, The tem b alsQ capab[e of arrailging ^ scis in aa order , 
and the tourth position ot the string should be an odd and creating transitions between assets, to create a concat- 
number, hut ls read as the numeral 8, it is likely that the is cnatcd strcam of audiovisual programming that has transi- 
fourth field contains the numeral 8 if it is also true that the lions mal appear to the viewer to have been edited by a 
fourth field cannot by design contain an even number. When human editor. Hven without tempo raForder, the assets con- 
such techniques arc combined with dynamically weighted tain sufficient transition information, and the system, con- 
general error correction, the scheme becomes tolerant of tains default transition choices, to permit assembly of pro- 
user inaccuracies and OCR errors. 20 gramming with fluid transitions without the intervention of 
For the presentation of video -superimposed characters, a a human editor. Numerous methods are available for order- 
static template could also be used which matches an accom- ing the assets in such a program. As discussed, assets may 
panying printed template. Alternatively, video-generated be ordered by using a template that imposes sequential 
characters could be derived dynamically from the character requirements on the assets, an expert system, a truth table, 
sequence used to identify the viewer. In this scenario, 25 or a human editor may decide the order of the assets. The 
mulliple-choice questions asked of the viewer would be attributes of the assets impose certain limitations on the 
presented on screen as would be the appropriate multiple- ordering of the assets. For example, it may not be possible 
choice answers, but the characters used to identify the for aesthetic reasons to place a certain clip immediately 
answers would be dynamically generated in response to the before or after a certain other clip. 

viewer or programming identification sequence. For 30 The individual assets may have demographic character- 
example, if an algorithm generated the output X, then the istics. For example, an audiovisual clip may feature actors of 
characters A, B, C and D would be generated next to four a particular ethnic group, geographic origin, age, or other 
multiple-choice answers appearing on screen. If the algo- demographic characteristic. As a result, the assets may be 
rithm generated an output of Y, the answers would be selected based on demographic search criteria, 
assigned the numerals 1,2, 3 and 4. Such a technique has the 35 The system may permit, in an on-line environment, 
advantage of allowing the video assets containing test mate- switching between a predetermined program and interactive 
riaJs to remain mutable while responding dynamically to the video. For example, the system may provide delivery of a 
needs of the error-correc ting/checking system. It will be predetermined audiovisual program so long as the viewer 
understood that the foregoing techniques may be employed does not seek to interact with Lhe system, e.g., by positioning 
with any visual, sonic or other cue. 40 a mouse pointer on the screen and clicking. At this point, the 

Various points of the systems that are believed by the system may select assets that are appropriate to the particu- 

inventor will now be emphasized. However, the enumera- lar information that the viewer appears to be seeking based 

tion of certain features believed to be novel should not be on the information on the screen at the time. The system will 

construed as implying that other features arc not novel. then generate suitable transitions and add those assets to the 

The system provides for a database of assets which must 45 programming. For example, the program may be a tour of a 

be arranged temporally in order to be used, but which does' three-dimensional model of the heart. When the user moves 

not dictate the temporal organization. Rather, the audio and a pointer or mouse to a particular portion of the screen, and 

video assets in the database are characterized by such clicks on the screen, the system may select, for example, 

parameters as content, aesthetic features, and suitability to assets incorporating more detailed information on certain 

viewers with certain demographic or psychographic charac- 50 portions of the heart corresponding to material on the 

terisLics. The user can select assets through a suitable query portion of the screen where the mouse was located when 

without attention to temporal organization. The organization clicked. The system applies suitable transitions to the assets 

of assets contrasts with the organization of assets in such and incorporates these assets into the programming interac- 

Pields as video games. In a video game, the permitted lively in response to mouse clicks or other suitable input 

temporal orders of the assets are predetermined by a strict 55 from the viewer. This interactive generation of programming 

branching system. Such a branching system defines pre- for passive viewing contrasts with existing systems that 

cisely which asset may be permitted to follow which under include a predetermined loop if the viewer does not provide 

all possible circumstances. The database of the present inputs. 

system may be termed amorphous, in contrast to the rigid The system may also be employed in an interactive 
organization of a database having a branching system. The 60 system that leads the viewer to select, a desired sequence, 
general principle can be extended to other types of assets The system may, for example, provide more interesting 
that require temporal organization in order to be used. For visual data in areas of the screen that lead to the selection of 
example, a database can be constructed for use with still material that the viewer should see, or provide smooth 
■ photographs to be arranged into a program with an audio transitions from audiovisual assets incorporated in the pro- 
track. Interactive assets, such as two-dimensional or three- 65 gram as a result of viewer input to audiovisual assets 
dimensional models and graphical user interfaces can also incorporated in the program to achieve a predetermined 
be catalogued in a database of this type. The present also purpose, e.g., to convey certain information to. the viewer. 



